Translation Lexicon Estimates from Non-Parallel Corpora Pairs

The estimation of translation lexicon probabilities from parallel corpora is well studied in statistical machine translation. Whenever parallel corpora are not available, it is still possible to obtain unsupervised estimates from pairs of monolingual, non-parallel corpora. In both cases the standard estimator is the Expectation-Maximization (EM) that aims at increasing the likelihood of the source corpus under the translation model. In this paper we study the utility of maximizing the joint likelihood of source and target corpora under a bi-directional translation model. A recently presented bi-directional estimator (BiEM [10]), which maximizes the joint likelihood, constitutes an instance of the EM algorithm. We show that Bi-EM reconciles the asymmetric statistics in the two corpora and leads to better lexicon estimates than standard EM. Our extensive experimental results show that relative to standard EM, our Bi-EM gives substantially better word-to-word translation results across variably related pairs of monolingual corpora.